Today we can just pick the right open-source tool, such as
cronR and taskscheduleR R packagesSource: thecodinglove.com
kid driving the car animgif?
It can be anything … just be consistent!
tidyverse style guideExtra hints:
lintr in a pull request builder / CI toolDecided to spend a couple hours coding in the hope of a modern and flexible logging engine in #rstats, then ended up thinking about and sketching the “Anatomy of a Logging” (tm) also writing docs for days 😜
— Gergely Daróczi (@daroczig) November 27, 2018
Looking forward to any feedback and comment! https://t.co/GbT1uXfNX9 pic.twitter.com/uXsnKfDxsi
roxygen2, potentially with markdownR CMD check frequently to check on your docspkgdownStart from scratch:
Contribute to an existing project:
library(logger)
log_threshold(TRACE)
log_info('Script starting up...')
#> INFO [2019-04-04 04:25:17] Script starting up...
log_info('The mtcars dataset has {nrow(mtcars)} rows!')
#> INFO [2019-04-04 04:25:17] The mtcars dataset has 32 rows!
log_eval(nrow(mtcars))
#> TRACE [2019-04-04 04:25:17] 'nrow(mtcars)' => '32L'cat demo.R
library(logger)
pkgs <- available.packages()
log_info('There are {nrow(pkgs)} R packages hosted on CRAN!')
for (letter in letters) {
lpkgs <- sum(grepl(letter, pkgs[, 'Package'], ignore.case = TRUE))
log_level(if (lpkgs < 5000) TRACE else DEBUG,
'{lpkgs} R packages including the {shQuote(letter)} letter')
}Rscript demo.R
INFO [2018-20-11 22:49:37] There are 13433 R packages hosted on CRAN!
DEBUG [2018-20-11 22:49:38] 6300 R packages including the 'a' letter
DEBUG [2018-20-11 22:49:38] 6772 R packages including the 'e' letter
DEBUG [2018-20-11 22:49:38] 5412 R packages including the 'i' letter
DEBUG [2018-20-11 22:49:38] 7014 R packages including the 'r' letter
DEBUG [2018-20-11 22:49:38] 6402 R packages including the 's' letter
DEBUG [2018-20-11 22:49:38] 5864 R packages including the 't' letterlibrary(logger)
log_layout(layout_glue_colors)
log_threshold(TRACE)
log_info('Starting the script...')
log_debug('This is the second log line')
log_trace('Note that the 2nd line is being placed right after the 1st one.')
log_success('Doing pretty well so far!')
log_warn('But beware, as some errors might come :/')
log_error('This is a problem')
log_debug('Note that getting an error is usually bad')
log_error('This is another problem')
log_fatal('The last problem')log_threshold(WARN)
log_formatter(formatter_sprintf)
log_layout(layout_json(c('time', 'node', 'user', 'pid', 'msg')))
t <- tempfile()
log_appender(appender_file(t))
log_debug('I am a low level log message that will not be printed with a high log level threshold')
log_warn(
'I am %s higher level log message that is very likely to be printed!',
letters[1])
cat(readLines(t))with_log_threshold(log_warn('booo'), threshold = ERROR)
log_with_separator('This message is worth a {1e3} words')
#> INFO [2019-03-29 19:00:15] =====================================================
#> INFO [2019-03-29 19:00:15] = This message is worth a 1000 words =
#> INFO [2019-03-29 19:00:15] =====================================================
log_with_separator('Some critical KPI down!!!', separator = '$')
#> INFO [2019-03-29 19:00:15] $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
#> INFO [2019-03-29 19:00:15] $ Some critical KPI down!!! $
#> INFO [2019-03-29 19:00:15] $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
log_with_separator(paste(
'A very important message with a bunch of extra words that will',
'eventually wrap into a multi-line message for our quite nice demo :wow:'),
width = 60)
#> INFO [2019-03-29 19:00:15] =================================
#> INFO [2019-03-29 19:00:15] = A very important message =
#> INFO [2019-03-29 19:00:15] = with a bunch of extra words =
#> INFO [2019-03-29 19:00:15] = that will eventually wrap =
#> INFO [2019-03-29 19:00:15] = into a multi-line message =
#> INFO [2019-03-29 19:00:15] = for our quite nice demo =
#> INFO [2019-03-29 19:00:15] = :wow: =
#> INFO [2019-03-29 19:00:15] =================================library(DBI)
con <- dbConnect(
RMySQL::MySQL(),
dbname = "shinydemo",
host = "shiny-demo.csa7qlmguqrf.us-east-1.rds.amazonaws.com",
username = "guest",
password = "guest")
query <- dbSendQuery(
con,
"SELECT Continent, COUNT(DISTINCT(Region)) FROM Country GROUP BY Continent")
res <- dbFetch(query)
dbClearResult(query)
dbDisconnect(con)“When I woke up the next morning, I had four emails and a missed phone call from Amazon AWS – something about 140 servers running on my AWS account, mining Bitcoin.” – Andrew Hoffman
SELECT
TO_HEX(CODE_POINTS_TO_BYTES([0xac, num2, num3, num4])) AS nonce
FROM
UNNEST(GENERATE_ARRAY(0, 255)) num2,
UNNEST(GENERATE_ARRAY(0, 255)) num3,
UNNEST(GENERATE_ARRAY(0, 255)) num4
WHERE
TO_HEX(REVERSE(SHA256(SHA256(CONCAT(FROM_HEX(
'000000204a4ef98461ee26898076e6a2cfc7c764d02b5f8d670832000000000000000000f99f5c4d5025979fcb33d245536a55b628d4564c075c0210cbbc941ad79fdbc5e491b55a494a5117'), CODE_POINTS_TO_BYTES([0xac, num2, num3, num4]) ))))) LIKE '000000000000000000%'library(DBI)
con <- dbConnect(
RMySQL::MySQL(),
dbname = "shinydemo",
host = "shiny-demo.csa7qlmguqrf.us-east-1.rds.amazonaws.com",
username = getOption("my_username"),
password = getOption("my_password"))But how to set those global options on the server?
con <- DBI::dbConnect(
conf$drv,
dbname = conf$dbname,
host = conf$host,
username = keyring::key_get("my_username"),
password = keyring::key_get("my_password"))Great for the single-desktop R user, but how to make use of it on a remote server?
But we still need someone to set up / deploy configuration.
But we still need to set up ~/.my.cnf:
[shinydemo]
user=guest
password=guest
database=shinydemo
host=shiny-demo.csa7qlmguqrf.us-east-1.rds.amazonaws.comAnd then .pgpass etc as well.
mysql_user <- readRDS('secrets/mysql_user.RData')
mysql_pass <- readRDS('secrets/mysql_pass.RData')
library(DBI)
con <- dbConnect(
RMySQL::MySQL(),
dbname = "shinydemo",
host = "shiny-demo.csa7qlmguqrf.us-east-1.rds.amazonaws.com",
username = mysql_user,
password = mysql_pass)Again, how to get those unencrypted RData files to the server?
library(secret); try(local_key(), silent = TRUE)
vault <- file.path(tempdir(), ".vault"); dir.create(vault); create_vault(vault)
key_dir <- file.path(system.file(package = "secret"), "user_keys")
alice_public_key <- file.path(key_dir, "alice.pub")
alice_private_key <- file.path(key_dir, "alice.pem")
add_user("alice", alice_public_key, vault = vault)
secret_to_keep <- c(password = "guest", username = "guest")
add_secret("secret_one", secret_to_keep, users = "alice", vault = vault)
secrets <- get_secret("secret_one", key = alice_private_key, vault = vault)
mysql_user <- secrets$my_username; mysql_pass <- secrets$my_password
con <- DBI::dbConnect(
RMySQL::MySQL(), dbname = "shinydemo",
host = "shiny-demo.csa7qlmguqrf.us-east-1.rds.amazonaws.com",
username = mysql_user, password = mysql_pass)But how to get the private key to a new server?
library(DBI)
con <- dbConnect(
RMySQL::MySQL(),
dbname = "shinydemo",
host = "shiny-demo.csa7qlmguqrf.us-east-1.rds.amazonaws.com",
username = Sys.getenv("my_username"),
password = Sys.getenv("my_password"))Again, how to set those env vars on the server?
conf <- config::get("shinydemo")
con <- DBI::dbConnect(
RMySQL::MySQL(),
dbname = conf$dbname,
host = conf$host,
username = conf$username,
password = conf$password)With the below YAML config:
With the below YAML config:
default:
shinydemo:
drv: !expr RMySQL::MySQL()
host: shiny-demo.csa7qlmguqrf.us-east-1.rds.amazonaws.com
username: guest
password: guest
dbname: shinydemoBut again, we have to get the YAML file to the server in a secure way :/
Source: AWS Encryption SDK
Source: AWS Encryption SDK
With the below YAML config:
sss
todo
wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
echo "deb https://pkg.jenkins.io/debian-stable binary/" | sudo tee -a /etc/apt/sources.list
sudo apt update
sudo apt install openjdk-8-jdk-headless jenkins
sudo netstat -tapen | grep java
Access Jenkins from your browser and finish installation
sudo cat /var/lib/jenkins/secrets/initialAdminPassword
crontab.guru
H => hash
rotation
sync to cloudwatch
todo
extended email (edit subject / body, attachement, inline images – even from docker) slack notification
https://stackoverflow.com/questions/8175912/load-multiple-packages-at-once/8176099#8176099
How it feels when you look into a new issue opened for some of your stale #rstats projects https://t.co/CvuED5eB5A pic.twitter.com/LGQC76bcOX
— Gergely Daróczi (@daroczig) December 11, 2015
https://github.com/rajgoel/reveal.js-plugins/tree/master/embed-tweet